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METHOD AHD APPARATUS FOE V™™?™' 3 
DEMAND REAL-TIME TELEVISION 

^g-PEFF.R ""™ ™ » ^TATFr. SPPT..ICATIQ H 

\ T he disclosure contained in this application^ ^related 

to U 1 patent application serial number <_ 

(attornV docket number 167,, filed simultaneously herewith 
and herek incorporated by reference. 

r .„ r „ r ^„ » thk D ISCL O SURE . 

1 Field of the Invention • 
Th e invention relates to video-on-demand systems and. 
15 more particularly, the invention relates to a method and 
apparatus for g eneratin g a real-time video stream for 
transmission through a video-on-demand system as wel^as 
simultaneously storing the video information r a format 
that facilitates demand television, i.e., a play bitstream, 
. „„,q a fast reverse bitstream that 

a fast forward bitstream and a fast revers 
are derived from the real-time video bitstream content. 

2. Description of the Background Art 

m a video-on-demand system such as the OnSet™ system 
manufactured by DIVA Systems Corporation, a ^llZJrl 
used for streaming video information to users (subscribers) 
of the system. The OnSet™ system is described in U.S. 
patent application serial number 08/984,710, filed December 
3 1997, and the file server is described in U.S. patents 
5 ' 67 1 377 and 5,581,778. The disclosures of thrs ^ 
application and these patents are incorporated herein by 
reference. The OnSet™ system contains service provider 
equipment coupled through an information distribution 
network to subscriber equipment. This system P--des 
subscribers VCR-like controls to enable a 

select information content, for example, a movie then play, 
fast forward, rewind, pause, or stop the selected movie. 
The subscriber enters control commands through the 
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subscriber equipment and the service provider equipment 
executes the co^nands to fulfill the purpose of the command, 
e g play, fast forward, rewind, stop or pause the movre. 
' ' A file server forms a portion of the service provider 
5 equipment and stores, for a given movie, a standard play 
stream (i.e., an MPEG-2 compressed video bitstream) , a fast 

j „ f __ r rev . r se play (rewind) stream. The 
forward stream and a fast reverse pj.=i 

fast forward and fast reverse streams are also MPEG-2 
compressed video, where the fast forward stream rs , 
L 0 compressed signal containing only every Nth frame of the 
uncompressed play video and the fast reverse stream xs a 
compressed signal containing every Nth frame of the 
uncompressed play video played backwards. To store the 
video information, each stream is divided into portions 
15 (e g N-byte segments of compressed data) and strxped onto 
a disk array. The disk array stores all the informatron 
that the file server can directly access. 

The process for generating the fast forward and fast 
reverse streams is performed in a non-real time manner such 
20 that the video content is pre-encoded and stored in the fxle 
server The process requires a frame by frame analysrs of 
the video stream to enable the frames to be encoded in an 
MPEG-2 compliant form, yet provide a fast forward and fast 
reverse effect when decoded. As such, the video sequence rs 
25 processed to extract every Rth frame (i.e., one of 

frames, where R is an integer greater than 1) to form a fast 
forward sequence and then the fast forward sequence rs 

^\ pnr a fast reverse stream, a sequence 
encoded (compressed). For a tast rever 

of frames in reverse order is created and encoded. This 
30 encoding process can not be used for producing fast forward 
and fast reverse streams in real time such that a real txme 
program can be encoded and stored for almost immediate use 
of VCR-like functions. 

Therefore, there is a need in the art for an improved 
3 5 encoding system for a video -on- demand system to ensure near 
real-time availability of fast forward and fast reverse 
functions and real-time availability of a high bit rate 
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video bitstream that, when decoded, produces a play 
sequence . 

QTTMMARY THE INVENTI ON 

The disadvantages associated with the prior art are 
overcome by the invention of a demand television system that 
simultaneously encodes a broadcast video frame sequence xnto 
a broadcast bitstream and a storage bitstream. The 
broadcast bitstream is broadcast to system subscribers as 
the sequence is encoded, while the storage bitstream is 
stored in an information server. The subscriber may, at any 
time during the broadcast, elect to review content of the 
broadcast that was previously displayed. Upon electing to 
5 review, the system transmits the storage bitstream to the 
subscriber in a pointcast manner. The storage bitstream 
contains, for example, a plurality of selectable types of 
bitstreams including fast forward, fast reverse and standard 
play As such, by requesting particular functions, the 
0 subscriber is provided with VCR- like functions for a 
broadcast program. 

To facilitate this functionality, a video encoder that 
simultaneously produces an MPEG-2 compliant fast forward, 
fast reverse and play bitstreams from a sequence of video 
>5 frames, e.g., 601-format video, as well as a real-time 
bitstream for real-time transmission to the user as a 
broadcast transmission. The encoder of the present 
invention contains a broadcast encoder and a storage 
encoder The broadcast encoder encodes the video frame 
30 sequence using, for example, a high bit rate encoder to 
ensure accurate encoding and transmission of sporting 
events While simultaneously the storage encoder subsamples 
the video sequence, extracts a plurality of frames from the 
video sequence and buffers the subsampled frames. 
35 Simultaneous with the subsampling and buffering, the storage 
encoder also encodes the source frames within a real-time 
encoder, e.g., an MPEG-2 encoder, to form a standard play 
bitstream for storage within the file server. 
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As the play bitstream is being encoded, the buffered 
frames are recalled fro, the buffer and coupled "a second 
real-time encoder. The second encoder forms both the fast 
forward and fast reverse bitstreams using a trme 
multiplexing technic^ wherein a group of Pictures (GOP) 
the fast forward stream can be formed, followed by the 
compression of the same GOP having the frames organized rn 
reverse order. As such, the compressed GOPs are represented 
by the fast forward and fast reverse bitstreams. The play, 
fast forward, and fast reverse bitstreams for each GOP are 
organized into a file and stored on the mass storage devrce 
(eg disk drive array) of the file server. 

' 'while the bitstreams that facilitate the VCR-like 
functions are being encoded and stored, the high bit rate 
encoded signal is broadcast to subscribers. As such the 
subscriber may watch an event (e.g., a sporting event) rn 
real-time, then elect to -rewatch- a previously viewed 
portion of the real-time event. The viewer merely depresses 
a -rewind" or reverse button on a remote control and the VOD 
, system accesses the stored event files and "plays- the fast 
reverse bitstream. As such, the viewer is provxded wrth the 
sense of a VCR rewind function. At an appropriate loca tron 
the viewer depresses a "play- button and the stored standard 
play bitstream is transmitted to the viewer. The viewer may 
5 -catch up- to the real-time event by either depressing a 
fast forward button or a -real-time" button. The fast 
forward button causes the stored fast forward bitstream to 
be transmitted to the viewer to provide a sense of a VCR 
fast forward mode and. when the fast forward bitstream 
,0 exhausts the available data, the system automatically 

switches back to the real-time bitstream. The "real-time 
button causes the system to instantly switch back to the 
real-time bitstream. 
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nEsnp-"""-"™ " F ™ R DRAWINGS 

Th e teachings o£ the present invention can be readily 
understood by considering the following ^axled^escrxptron 
in conjunction with the accompanying drawings, xn whxch 

VlG 1 is a block diagram of a video-on-demand system 
that operates in accordance with the present 

FIG . 2 depicts a block diagram of the encoder of the 

^^T^trates the method of the present invention 
used to produce a fast forward bitstream; 

FIG 4 illustrates the method of the present xnventxon 
used to produce a fast reverse bitstream; and 

PIG 5 illustrates the interaction model between the 
service provider equipment and subscriber equipment. 

To facilitate understanding, identical reference 
numerals have been used, where possible, to designate 
identical elements that are common to the fxgures. 

0 niT»TT,m pESrRTPTION 

FIG 1 depicts a high level block diagram of the 
illustrative information distribution system 100 that 
n o^orates the present invention. The system 100 contains 

5 service provider equipment 102, a communications network xn 
rfoJof a cable transport subsystem 10* 
equipment 106,. where n is an integer greater than 0. The 
service provider equipment 102 contains an xnformatxon 
IZi: tt. which is typically a parallel processing computer 

30 containing at least one central processing un, £ 1 and 
associated memory 112. U.S. patent number 5 671 377 

35 the capacity of informatxon server 108. The server 1 

interacts with the data storage devxce 114 (e.g., a dxsk 
drive array, that generally stores the 

information (e.g., video data, that is transmxtted dxrectly 
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to the subscriber equipment 106 as well as recalled and 
downloaded to the subscriber equipment 106. The video data 
is produced by an encoder 200 as two streams: the first is 
a stream containing a real-time bitstream of encoded video 
5 information that is to be broadcast to the subscribers 

(referred to herein as the broadcast stream) . the second is 
a stream containing a non-real-time bitstream of encoded 
video information that is stored by the information server 
to facilitate VCR-like functions (referred to herein as the 

10 storage stream). _ 

Additionally, within the service provider equxpment 102 
is a video session manager 122 that provides session control 
of the information flowing to and from the server 108. The 
video session manager 122 contains its own central 
15 processing unit (CPU) 124 and associated memory 126 that 
provides functionality for the graphical user interfaces 
through which the consumer interacts with the system. The 
CPU 124 is part of a session control manager 125 that 
controls a plurality of modems 127 that facilitate 
2 0 communication with the subscriber equipment. Other 

subsystems of the service provider equipment include a 
network manager 142 and a back office subsystem 144. These 
subsystems maintain certain databases of information that 
enable the system to accurately control system access, 
25 subscription package definitions, and subscriber/ consumer 
profile and billing. 

The information server 108 is coupled to the video 
session manager 122 via data path 116. synchronization clock 
path 118, and control path 120. The server 108 provides 
30 data streams that are destined for consumers on path 116 and 
a synchronization clock on path 118. The specific data 
streams are provided in response to requests for information 
(e.g., menu applets, video programs, and other content 
material) from the video session manager 122 on path 120. 
3 5 These data streams are packetized and modulated onto a 
carrier that is compatible with the transmission 
requirements of the network 104. 
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The video session manager 122 accomplishes all of the 
transmission interface retirements of the system 100 as 
well as provides graphical user interface support, 
specifically, the video session manager 122 is coupled 
* ^ n ^modems 127 to subscriber ec.ipment via a orward 
Ql 1-^9 a forward command channel 133 and a 
information channel a IOIwaiU , 

b L channel 134. Ml three of these channels are supported 
by the cable transport subsystem 104. The vxdeo session 
lager 122 contains a modulator for modulating the server 
data streams onto one or more carrier fre^encie or 
transmission on the forward information channel 132. 
Additionally, modems 127 within the video session manager 
122 send control information via the forward command channel 
and receive control information via the back — 
5 Moreover, a conventional cable television signal source 128 
Ts Anally coupled to the forward information channel vra 
a signal coupler 130. In operation, the video session 
manager 122 responds to recasts from the subscriber 
equipment 106 for interactive menus and data streams by 
„ revesting the server 108 to provide such information, then 
—eating that information to the -uesting subscriber 
eguipment 106. The video session manager 122. as *™ sd 
b elow, also ensures that the subscriber equipment 106 
authorized to receive the revested information. 

The cable transport subsystem 104 can be any one of 
number of conventional broad band communications networks 
that are available such as a fiber optic network, a 
telephone network, existing cable television network and the 
Uke For example, if the network is a hybrid fiber-coax 
3„ network, the transport technique used in both forward 

channels may be modeled after the moving pictures expert 
group (MPEG) transport protocol for the transmission of 
Tdeo data streams. Xn general, the transport mechanism or 
both the forward channels and transport information to the 
35 subscriber eguipment must be able to carry "f ^^on* 1 , 
asynchronous packetized data such as that defined in the 
MpS video and audio signal transmission protocol, and the 
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like. There are a number of such transport protocols 
available. 

The subscriber equipment 106 receives the requested 
data streams as well as broadcast streams from the forward 
information channel, demodulates the streams and processes 
them for display on the display device 140 (e.g., a 
conventional television,. In addition, the terminal 136 
accepts commands from a remote control input devrce 13 8 or 
other input device to facilitate consumer interaction with 
the system. These commands are formatted, compressed, 
modulated, and transmitted through the network 104 to the 
video session manager 122. Typically, this transmission is 
accomplished through the back channel 134. These commands 
are preferably transmitted through the same network used to 
; transmit information to the subscriber equipment. However, 
the back channel coupling the subscriber equipment to the 
server may be a separate network, e.g., a forward 
information channel through a television cable network and a 
back channel through a telephone network. The telephone 
0 network could also support the forward control channel The 
video session manager 122 interprets each command set from 
the terminal through the back channel and instructs the 
information server to perform certain functions to implement 
the consumer/ subscriber request. 
. 5 FIG 2 depicts a block diagram of the encoder 200 

comprising a broadcast encoder 250 and a storage encoder 
255 The broadcast encoder a source video sequence in a 
conventional manner, i.e., compressing the source video 
sequence in real-time as the frames are input to the 
3 0 encoder. For example, this encoder may be a high speed 
encoder such as an 8 Mbps MPEG-2 encoder that accurately 
encodes such difficult to compress programming such as 

sporting events. 

The storage encoder 252 comprises a first encoder 202, 
35 frame subsampler 204, a frame buffer 206. a second encoder 
208 and a controller 210. The first encoder 102 encodes a 
source video sequence in a conventional manner, i.e., 
compressing the source video sequence in real-time as the 
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frames are input to the encoder. The second encoder 208 
operating in conjunction with the subsampler 204 and the 
buffer 206 encodes a subsampled version of the source video 
sequence to form a fast forward and fast reverse bitstreams 
5 (collectively referred to herein as trick play bitstreams or 
trick play streams) . The first encoder (the play stream 
encoder 202) contains a real-time MPEG-2 encoder that 
produces an MPEG- 2 compliant, compressed video bitstream (a 
play stream) from a sequence of 601-format video frames. 
L0 The second encoder (the trick play stream encoder 204) is 
also an MPEG-2 real-time encoder 212. 

For the following discussion, frame numbering is used 
to describe the temporal order in which frames occur m 
source material where 1 is the first frame in the source 
15 material frame sequence and 2 is the second frame and so on. 
For the following description, R is an integer and defines a 
play-back speed multiplier which has significance m the 
trick play processes. The speed multiplier R is a variable 
that is established by the controller 210. Throughout this 
20 disclosure, the exemplary trick play streams are fast 

forward and fast reverse. Of course other forms of trick 
play streams may be generated using the encoder 200 of the 
present invention. If the multiplier R equals 2, playback 
(decoding) of a fast forward stream is twice normal single 
25 speed If R equals 3, playback is three times normal play 
speed and so on. Generally, the trick play streams include 
a nine times fast forward stream and a nine times fast 
reverse stream. Alternatively, 32 times normal fast forward 
and fast reverse streams are also available. The term 
3 0 source is used to describe the uncompressed video material 
(601-format video) from which fast forward and fast reverse 
bitstreams are generated. 

Using the encoder 200, a fast forward MPEG video 
bitstream is generated from an uncompressed video source 
35 such that when the stream is played back linearly through a 
standard MPEG compliant video decoder, the resulting imagery 
contains every Rth frame of the original video sequence^ As 
such, the display of the sequence has a fast forward effect. 



A fast reverse stream is simultaneously produced by the 
storage encoder 252 such that when linearly played back 
through a an MPEG compliant video decoder, the decoded 
stream produces a sequence of frames that play in reverse 
relative to the original frame sequence. 

The play bitstream is formed using a conventional 
MPEG-compliant encoder 202 that compresses a plurality of 
frames (N frames that form a group-of -pictures (GOP) ) - To 
form the trick play streams having N frame GOPs , the 
subsampler 204 extracts one out of every R consecutive 
frames (arranged in increasing time code order) from the 
source sequence. The selected N frames are buffered in the 
frame buffer 206. For example, if R is two and the buffer 
stores N frames, the buffer stores frames 1 , 3 , 5 , 7 , 9 , and 
, so on up to frame 2N + 1. The buffer 206 stores N successive 
frames that define a GOP for the trick play bitstreams . 

To produce a fast forward stream, the N frames from the 
buffer 206 are recalled under the control of the controller 
210 and coupled sequentially into the real-time encoder 208. 
0 The output is a compressed bitstream representing a fast 
forward GOP. 

To produce a fast reverse stream, the N frames from the 
buffer 206 are recalled under the control of the controller 
210 and are coupled in reverse time order into the encoder 
5 208 Although two encoders could be used to produce each of 
the trick play streams independently, as long as R equals 
two or more, a single encoder can be used to produce both 
trick play tracks using time multiplexing technique. To 
perform the multiplexed encoding, the fast forward GOP xs 
30 encoded first, then the fast reverse GOP. After the buffer 
is filled with another N frames, the GOP pair is again 
encoded and so on. The recall order of the frames from the 
buffer and the multiplexing process are controlled by 
controller 110 by addressing the buffer in forward order for 
3 5 the fast forward GOP and in reverse order for the fast 
reverse GOP . 

The forward and reverse GOPs contain sequence start and 
sequence end codes such that, when stored in a mass storage 
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device along with the standard play stream the bitstreams 
are clearly delimited. 

■ The forward and reverse GOP bitstreams are stored wrth 
the play stream in a storage medium such as a disk drive 
array or magneto-optical disk. The fast forward stream is 
written sequentially, one GOP bitstream at a time, from the 
lowest address to the highest address. The averse 
bitstream is stored in reverse order from highest address to 
lowest address. The last byte in the reverse OOP i, J placed 
in the highest address space of the reverse target file. As 
such the reverse GOP precedes, in address space, the first 
frame of the immediately previously written reverse GOP 

The entire source material frame sequence is encoded m 
this manner to simultaneously produce a play, a fast 
forward, and fast reverse streams in real-time. 

FIG 3 illustrates the process by which a fast forward 
stream is produced. Sequence 3 00 represents the RN frames 
that have been subsampled from a video frame sequence at a 
one of R rate from a video frame sequence and stored m the 
0 buffer (e.g., R=2, then 2N frames are subsampled to form a 
trick play GOP) . The integer N is the number of frames that 
are encoded into each GOP of the standard play stream 
These buffered frames are recalled from memory, encoded and 
then associated with a normal play stream having a GOP of N 
5 frames. The encoder (at step 302, repetitively produces a 
sequence of GOPs 304 that are stored in increasing addresses 
in memory as illustrated in sequence 306. The ^st GOP 
(G0 P 0) contains frames 1 to RN + 1, the second GOP (GOP 1) 
contains frames R(N + 1, + 1 through 2RN+1, the third GOP (GOP 
30 2) contains frames 2R(N + 1) + 1 through 3RN.1, and so on. 
generalize, a GOP contains frames GR(N + D + 1 through 
(G+DRN+l, where G is the GOP number (e.g., 0, 1, 2, 3, " 
R is the subsampling rate, and N is the number of frames m 

a standard play GOP. 

PXG 4 illustrates the process by which a fast reverse 
stream is produced. Sequence 400 represents the RN frames 
that have been buffered after subsampling at one of R rate 
{e g R=2 in FIG. 4) . At step 402, these frames are 
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selected fro- the buffer in reverse order as -"-"ated at 
404 The reverse order frames are encoded, at step 406, to 
produce a sequence of GOPs 408 that are stored Reverse 
time order in memory as illustrated in sequence 410. 

Returning to FIG. 2, the real-time broadcast encoder 
250 is for example, a high data rate encoder (e.g., 
producing an 8 Mbps data rate MPEG bitstream) from a 
real-time television broadcast. The source video «. for 
example, a television feed of a sporting event The source 
video either arrives at the encoder in a frame-based drgrtal 
video format, such as 601 video, or is converted from some 
other format into a frarr.e-based video format prior to the 
encoder. The real-time broadcast encoder operates 
simultaneously with the storage encoder such that the real- 
3 time broadcast bitstream is coupled to the ^nsmissron 
system as the storage bitstreams are being stored m the 
information server's data storage. 

Returning to FIG. 1 and simultaneously referring to the 
flow diagram 500 of FIG. 5, The source video is encoded as a 
0 broadcast stream in step 502 and as a storage stream rn step 
504 As described above, the storage bitstream is stored xn 
the information server memory at step 506, while the 
broadcast bitstream is transmitted to the subscribers at 
step 508. To facilitate broadcast of the broadcast 
,5 bitstream, the broadcast bitstream is coupled into path 116 
' to the video session manager 122. The video session manager 
122 multiplexes the broadcast bitstream into a transport 
stream along with all the other multimedia signals, control 
signals and the like that are transmitted through the cable 
30 transport subsystem 104 to the subscriber equipment 106 
step 510, the subscriber terminal 136 demodulates, 
demultiplexes and decodes the broadcast bitstream for 

real-time display. 

To facilitate VCR-like functions with respect to the 
3S broadcast bitstream, the subscriber terminal 136 can request 
the stored bitstreams corresponding to the broadcast 
bitstream by manipulating the input device 138 (step 512, rn 
the same manner as the device is used to control the drsplay 
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of any other video asset. The subscriber, at any 
elect to review a portion of the program that has already 
been watched. As such, the subscriber manipulates the xnput 
device 138 (step 512) such that, at step 514, a -rewind- or 
5 fast reverse command is sent from the subscriber termxnal 
136 to a modem 127 requesting rewind. The session control 
manager 125 then instructs, at step 516, the information 
server 108 to recall the fast reverse stream assocxated wxth 
the broadcast bitstream and send, at step 518, the fast 
10 reverse stream to the requesting subscriber, i.e., the 
transmission to the subscriber is now changed from a 
broadcast transmission to a pointcast transmission. At step 
520 the subscriber terminal demodulates, demultiplexes and 
decodes the transport stream carrying the reverse play 
15 stream such that the display 140 depicts the broadcast video 
running quickly backwards at some rewind rate, e.g., nxne 
times standard play. 

At some point during the -rewinding- of the video 
fly sequenk the subscriber m ay, at step 522, elect to -play" 
the videisequence. By selecting the -play- button on the 
input device, the subscriber terminal sends, at step 524, a 
request to \e session control manager 125. The sessron 
control m anag\l25 then requests, at step 526, the standard 
play stream to\ sent, at step 52 8, to the subscriber 
starting at the location at which the video sequence had 
been -rewound", i.\ whatever frame number the subscriber 
was viewing at the tiW the play button was depressed wrth 
be the starting frame ^ber that is sent to the subscriber 
in the play stream. At b\p 530, the play stream is then 
demodulated, demultiplexed,\nd decoded such that the ^ 
display in the subscriber's h^me transitions from rewxnd to 
play The subscriber may then wa\ch the program from thrs 
point forward or may request anothe\ trick play function 

For example, at step 532, the subscriber selects a fast 
forward function to move through the video quickly, e.g., 
nine times standard play speed. If the subscriber elects to 
fast forward through the video, the subscriber termrnal once 
again sends, at step 534, a request to the session control 
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manager 125 which, in turn, requests, at step 536, the 
information server to recall the fast forward stream 
starting at the frame (or near the frame) that the user was 
then watching in the play stream. The session control 
manager 125 sends, at step 538, the fast forward stream to 
the subscriber. At step 540, the fast forward stream xs 
then demodulated, demultiplexed and decoded to present a 
fast forward display. This stream will be transmitted until 
there is no longer any fast forward data, i.e., the storage 
bitstream catches up with the broadcast stream and an end of 
file (EOF) indicator is reached. At that occurrence, the 
subscriber terminal, at step 542, automatically switches 
the subscriber back to the broadcast stream. At step 544 
the subscriber terminal 136, demodulates, demultiplexes and 
, decodes the broadcast stream. 

An alternative to requiring the subscriber to fast 
forward to catch up to the broadcast stream is to provide a 
"catch up" button that, when depressed, causes the 
subscriber terminal to instantly transition from decodxng 
0 the storage bitstream to decoding the broadcast bitstream. 
This button may be a hardware button on the input devxce or 
a software button that is displayed on the televxsxon 

display. _ 

Although various embodiments which incorporate the 
5 teachings of the present invention have been shown and 
described in detail herein, those skilled in the art can 
readily devise many other varied embodiments that stxll 
incorporate these teachings. 



